R is command-line only while RStudio is a GUI ((graphical user interface)) version of R.

RStudio includes the following:

• Script separate from command-line (left-hand screen)
• Lists your variables (upper, right-hand corner)
• Manual and an extensive help function
• Easy install of new packages
• Plots are shown within RStudio

If you work with linux or want to start R from the terminal then open your terminal, change your directory to the R_exercises folder and then just type R. Then, you should see something like this:

We can use a similar way to connect to R on our servers, i.e ada. For this to work you can use the code below (just remove the # symbol).

#Ask what R version we have
#R.version

#start R
#R

#exit R
#q()

General

One nice way to commend code is combine R-code with informative text in R markdown format (rmd). R Markdown supports dozens of static and dynamic output formats including: HTML, PDF, MS Word, Beamer, HTML5 slides, Tufte-style handouts, books, dashboards, shiny applications, scientific articles, websites, and more.

The R markdown file specifies code chunks which will be executed in R (or python or bash) and plain text which will be written to the report as is. A report is created by rendering the file in R, then the R-code is executed and the results are merged in a pdf or html output.

How you create a Rmarkdown document in R:

• Open RStudio
• press “File/new File/R markdown”.

This will create an R markdown file that already contains some R code and text. You can also open this document (the rmd file) in RStudio and see how the code looks.

You can also open a new file in any text editor and save it with the .rmd extension.

R code in R markdown

• The R-code is embedded inbetween the {bash}** and ** symbols.

In Rstudio, on the top-right position of such a section you will find three symbols. Pressing the middle one will run all code chunks above,while the right symbol will run the current R-chunk.

An important menu-button is the Knit button at the top, left-hand corner of RStudio. Pressing this button will create the final, rendered document (i.e. a HTML or PDF)

Text in markdown

• Headings are defined with “#”, or “##”, or “###” for first, second and third level.
• Lists are created by using “*” for (bullets) and “1”, … for numbered lists.
• A lot of other options are possible, more information on these can be found in the following documents.

Markdown options

• include = FALSE — prevents code and results from appearing in the finished file. R Markdown still runs the code in the chunk, and the results can be used by other chunks.
• echo = FALSE —prevents code, but not the results from appearing in the finished file. This is a useful way to embed figures.
• message = FALSE —prevents messages that are generated by code from appearing in the finished file.
• warning = FALSE —prevents warnings that are generated by code from appearing in the finished.
• fig.cap = “…” —adds a caption to graphical results.

Other info on Rmarkdown

You can find some more information on Rmarkdown here:

• The Rmarkdown website: https://rmarkdown.rstudio.com
• You can find more R cheat sheets on https://rstudio.com/resources/cheatsheets/
• Another info sheet with info how to modify code: https://www.rstudio.com/wp-content/uploads/2016/03/rmarkdown-cheatsheet-2.0.pdf

If you have issues, there is a lot of help online

1. Online you can check:

• www.r-project.org stack overflow many more

2. Inside of R:

help(mean)
?mean

The workspace is your current R working environment and includes any user-defined objects (i.e. vectors, matrices, data frames, lists, functions). At the end of an R session, the user can save an image of the current workspace that is automatically reloaded the next time R is started.

• From now on the code is shown in the grey area.
• If you see a line symbol, such as [1] then this shows the result of running our code.
• For example, first we run the code getwd() and the second box shows us the results.

#print the current working directory
getwd() 

[1] "/Users/ninadombrowski/Desktop/WorkingDir/Github_Workflows/R_tutorial"

#list the objects in the current workspace
ls()   

[1] "encoding"    "growth_data" "input_file"  "out_dir"    

The directory from which you work is usually first set from where you start R. But it can be re-set to find your data more easily. Ideally, you make one wdir per project and define the path in the script (see later below). It is recommended to have a similar format for these project folders, i.e. consider to create subfolders for input and output files. From the wdir you set you can load files using absolute and relative paths.

An example would be something with a structure like this:

In this example you see that we have 4 projects, and in each folder we have the R_script and folders for the required input and required output files. Also useful to have is a text file with the session info and if you plan to share this with others it is also good to have a README file that provides some background on the analysis done.

Options to see the working dir and set the working directory in R are:

#print your wdir
getwd()

#setting your wdir
setwd(getwd())

Packages are a collection of functions and tools that can be added to R that are often contributed by the community. Careful: there might be incompatibilites and packages are updated frequently but updating can break dependencies.

You need to install packages and load them EVERY TIME you want to use them. Therefore, ideally add them at the beginning of your scripts.

Installing packages

We have two ways to install packages:

1. Via the console by typing:

install.packages("package-name")

This will download a package from one of the CRAN mirrors assuming that a binary is available for your operating system. If you have not set a preferred CRAN mirror in your options(), then a menu will pop up asking you to choose a location.

Use old.packages() to list all your locally installed packages that are now out of date. update.packages() will update all packages in the known libraries interactively. This can take a while if you haven’t done it recently. To update everything without any user intervention, use the ask = FALSE argument.

2. Using R studio:

Go to the lower right hand-side window and click on packages and then install. Find the packages your are interested in.

Notice: If libraries come with their own data (i.e. example tables), then the data needs to be loaded separately. I.e. via data(cars) to load the cars data from the cars package.

Loading packages into your current R session

R will not remember what libraries you have loaded after you closed R. Therefore, you need to load necessary libraries everytime. Here, we will load some libraries that are usually quite helpful and it is recommended to make the libraries you load part of each of your scripts. For example like this:

#some example packages needed for Rmarkdown
library(knitr)
library(kableExtra)

The <- symbol assigns a value to a variable,

General rules for the syntax R uses:

• R is case sensistive
• If a variable exists, it will overwrite it with a new variable without asking
• If you work with characters, i.e. words like ‘hello’, then this need to written with quotes around it: “hello” (this will become clearer below)
• ls() shows all the variables that are known by the system at the moment
• you can remove indiv. objects with rm() and remove all objects rm(list=ls())

For example:

We can store our path for the working directory in a variable, which is useful if we want to re-run our script from the same folder.

Notice set the path to where ever you download the tutorial.

#store a path (a character string) to a variable with the name wdir
wdir <- "~/Desktop/WorkingDir/Github_Workflows/R_tutorial"

#print variable to screen to see if we stored the right thing in our variable
wdir

[1] "~/Desktop/WorkingDir/Github_Workflows/R_tutorial"

#use our new  variable to set our wdir to the correct path
setwd(wdir)

We can store more or less everything in a variable and use it later. For example, we can store numbers and do some math with them:

#store some numbers
x <- 1
y <-4

#do some simple math with the numbers we have stored
x+y

[1] 5

Functions are build in code that we can use to make our life easier, i.e. we can calculate lengtsh of vectors, do math or do statistical analyses. Base R already knows many useful functions but loading new packages greatly increases our repertoire. A list of most used functions can be found here: http://www.statmethods.net/management/functions.html

A function consists of:

1. Function name
2. Arguments (optional, some might be set with a default) = control how exactly the function behaves
3. Body of the function = defines what the function does

As an example lets test some simple functions: print and log:

#use the print function
print(3+5)

[1] 8

#use the log function
log(10)

[1] 2.302585

Call the default values of a function

Every function commes with a set of arguments that you can set but that usually also have some default values. In R Studio you can easily access all those details with the help function.

• ? allows us to first of all check exactly what a function is doing. If you scroll down to the bottom of the help page you also get some examples on how to use a function.
• More specifically the help function also allows us to get details on the arguments of a function.
• For example, if we check the help page of read.table we see that by default this function does not read in a header and if we want to provide a header we have to change that argument.

#let's check what **log** is doing
?log

#lets check the default arguments of a function
?read.table

Other useful functions:

To work with any kind of data we need to first read the data into R to be able to work with it.

For tables, there are some things to be aware of:

• It matters what separator our table uses to specificy individual columns. I.e. some programs store data using commas while others use tab as default delimiter. By default R assume we use a tab, but we can change this behaviour when we read in the table.
• Do not have any hash symbols (#) in your table. R will read this as a commented cell and not read the table from that point onward
• Avoid empty cells, as these sometimes can mess up your data.

For now, let’s read in the table with our growth data and store it under the variable name growth_data

Notice, to read in the file GenomeInfo.txt we need to direct it to the correct path as we do not have the file in the working directory but in a subdirectory.

Options that are good to leep in mind when reading in a table:

• sep = define that our field separator is a tab. A tab is written like this /t. If your data is using a space or comma, you can change that here.
• header = tell R that our data comes with a custom header (the first row in your datafra)
• quote = deals with some annoying issue with data formatting in excel files

General notice:

• To view data, the head() command is extremely practical, use it always when you modify data to check if everything went alright
• dim() is another useful function that displays the dimensions of a table, i.e. how many rows and columns we have. Again, this is useful to verify our data after we have transformed it to check if everything went alright.
• colnames() allows to only we the column names
• rownames() allows to only we the row names. Usually these are numbers, but we can also add anything else into the rows.

#read in data
timecourse <- read.table("Input_files/Timecourse.txt", sep="\t", header=T,  quote = "")

#check the first few lines of our data
kable(head(timecourse), format='markdown')

#check the dimensions of our data
dim(timecourse)

[1] 627   5

#check the column names
colnames(timecourse)

[1] "SampleID"   "Nutrient"   "Condition"  "Timepoint"  "Rootlength"

#check the row names
rownames(timecourse)

  [1] "1"   "2"   "3"   "4"   "5"   "6"   "7"   "8"   "9"   "10"  "11"  "12" 
 [13] "13"  "14"  "15"  "16"  "17"  "18"  "19"  "20"  "21"  "22"  "23"  "24" 
 [25] "25"  "26"  "27"  "28"  "29"  "30"  "31"  "32"  "33"  "34"  "35"  "36" 
 [37] "37"  "38"  "39"  "40"  "41"  "42"  "43"  "44"  "45"  "46"  "47"  "48" 
 [49] "49"  "50"  "51"  "52"  "53"  "54"  "55"  "56"  "57"  "58"  "59"  "60" 
 [61] "61"  "62"  "63"  "64"  "65"  "66"  "67"  "68"  "69"  "70"  "71"  "72" 
 [73] "73"  "74"  "75"  "76"  "77"  "78"  "79"  "80"  "81"  "82"  "83"  "84" 
 [85] "85"  "86"  "87"  "88"  "89"  "90"  "91"  "92"  "93"  "94"  "95"  "96" 
 [97] "97"  "98"  "99"  "100" "101" "102" "103" "104" "105" "106" "107" "108"
[109] "109" "110" "111" "112" "113" "114" "115" "116" "117" "118" "119" "120"
[121] "121" "122" "123" "124" "125" "126" "127" "128" "129" "130" "131" "132"
[133] "133" "134" "135" "136" "137" "138" "139" "140" "141" "142" "143" "144"
[145] "145" "146" "147" "148" "149" "150" "151" "152" "153" "154" "155" "156"
[157] "157" "158" "159" "160" "161" "162" "163" "164" "165" "166" "167" "168"
[169] "169" "170" "171" "172" "173" "174" "175" "176" "177" "178" "179" "180"
[181] "181" "182" "183" "184" "185" "186" "187" "188" "189" "190" "191" "192"
[193] "193" "194" "195" "196" "197" "198" "199" "200" "201" "202" "203" "204"
[205] "205" "206" "207" "208" "209" "210" "211" "212" "213" "214" "215" "216"
[217] "217" "218" "219" "220" "221" "222" "223" "224" "225" "226" "227" "228"
[229] "229" "230" "231" "232" "233" "234" "235" "236" "237" "238" "239" "240"
[241] "241" "242" "243" "244" "245" "246" "247" "248" "249" "250" "251" "252"
[253] "253" "254" "255" "256" "257" "258" "259" "260" "261" "262" "263" "264"
[265] "265" "266" "267" "268" "269" "270" "271" "272" "273" "274" "275" "276"
[277] "277" "278" "279" "280" "281" "282" "283" "284" "285" "286" "287" "288"
[289] "289" "290" "291" "292" "293" "294" "295" "296" "297" "298" "299" "300"
[301] "301" "302" "303" "304" "305" "306" "307" "308" "309" "310" "311" "312"
[313] "313" "314" "315" "316" "317" "318" "319" "320" "321" "322" "323" "324"
[325] "325" "326" "327" "328" "329" "330" "331" "332" "333" "334" "335" "336"
[337] "337" "338" "339" "340" "341" "342" "343" "344" "345" "346" "347" "348"
[349] "349" "350" "351" "352" "353" "354" "355" "356" "357" "358" "359" "360"
[361] "361" "362" "363" "364" "365" "366" "367" "368" "369" "370" "371" "372"
[373] "373" "374" "375" "376" "377" "378" "379" "380" "381" "382" "383" "384"
[385] "385" "386" "387" "388" "389" "390" "391" "392" "393" "394" "395" "396"
[397] "397" "398" "399" "400" "401" "402" "403" "404" "405" "406" "407" "408"
[409] "409" "410" "411" "412" "413" "414" "415" "416" "417" "418" "419" "420"
[421] "421" "422" "423" "424" "425" "426" "427" "428" "429" "430" "431" "432"
[433] "433" "434" "435" "436" "437" "438" "439" "440" "441" "442" "443" "444"
[445] "445" "446" "447" "448" "449" "450" "451" "452" "453" "454" "455" "456"
[457] "457" "458" "459" "460" "461" "462" "463" "464" "465" "466" "467" "468"
[469] "469" "470" "471" "472" "473" "474" "475" "476" "477" "478" "479" "480"
[481] "481" "482" "483" "484" "485" "486" "487" "488" "489" "490" "491" "492"
[493] "493" "494" "495" "496" "497" "498" "499" "500" "501" "502" "503" "504"
[505] "505" "506" "507" "508" "509" "510" "511" "512" "513" "514" "515" "516"
[517] "517" "518" "519" "520" "521" "522" "523" "524" "525" "526" "527" "528"
[529] "529" "530" "531" "532" "533" "534" "535" "536" "537" "538" "539" "540"
[541] "541" "542" "543" "544" "545" "546" "547" "548" "549" "550" "551" "552"
[553] "553" "554" "555" "556" "557" "558" "559" "560" "561" "562" "563" "564"
[565] "565" "566" "567" "568" "569" "570" "571" "572" "573" "574" "575" "576"
[577] "577" "578" "579" "580" "581" "582" "583" "584" "585" "586" "587" "588"
[589] "589" "590" "591" "592" "593" "594" "595" "596" "597" "598" "599" "600"
[601] "601" "602" "603" "604" "605" "606" "607" "608" "609" "610" "611" "612"
[613] "613" "614" "615" "616" "617" "618" "619" "620" "621" "622" "623" "624"
[625] "625" "626" "627"

Useful comments:

Sometimes we have to deal with really large data that take long to load with read.table. The function fread() from the data.table package is a very nice alternative. Below, we will explain how to read in R libraries to use additional functions.

This script always uses kable for plotting, which is just to make it visually attractive in html. Whenever you see a function using kable simply replace it with the head() function.

Now, if we would have modified the table we might want to store it on your computer. We can do this using write.table() and below we use a different output directory. Notice, we always start from the location we set as working directory.

write.table(timecourse, "Output_examples/timecourse_changed.txt",  sep = "\t", row.names = T, quote =F)

• sep –> we define what delimiter we want to use
• row.names = T –> we want to include whatever is in the rownames
• quote = F –> we do not want R to add any quotes around our columns.

In R, all types of data are treated as objects. As such objects are units that we work with, i.e. data and functions.

Roughly we distinguish between:

• Data types = For example: numeric, characters, factors, … You can identify your data type with the function typeof()
• Data structures = For example: vector, matrix, list. You can identify the structure with class() or is()
• str() = defines what the indiv. components of your data are

Everything that exists is an object. Everything that happens is a function call.

Below, we introduce all these differnt types of objects.

A vector is a collection of items of the same type (i.e characters, numbers). You can read in numbers and characters into the same vector, however, the number will be then seen as a character if you mix different classes.

#lets create a random vector
a_vector <- c(2, 3, 5, 7, 1) 

#show the vector we just created
a_vector

[1] 2 3 5 7 1

#asking how long your vector is
length(a_vector)

[1] 5

Vector indexing

If we want to only retrieve part of the data stored in a vector we can create a subset using the index as shown below.

• square brackets [] = allow us to retrieve certain elements of a vector, i.e. [3] retrieves the 3rd element
• we can combine c() and [] if we want to retrieve several elements of a vector.

#retrieve the third element stored in a vector
a_vector[3]

[1] 5

#retrieve the 1st and 3rd element by combining ``c()`` and []
a_vector[c(1,3)]

[1] 2 5

#retrieve the 1-3rd element
a_vector[c(1:3)]

[1] 2 3 5

#we can also add vectors of the same length together
x <- c(1,2,3,4)
y <- c(1,2,3,4)

#and now we can combine our vectors
x + y

[1] 2 4 6 8

Beware: If we add two vectors of different length, the shorter vector is duplicated. This only works if the shorter vector is proportional to the longer one

#adding vectors of different lengths
x <- c(1,2)
y <- c(1,2,3,4)

#and now we can combine our vectors
x + y

[1] 2 4 4 6

Another way to extend vectors is:

• append() –> Add elements to a vector.

#add another datapoint to our vector
a_vector <- append(a_vector, 13)
a_vector

[1]  2  3  5  7  1 13

#add +1 to all our four numbers
a_vector <- a_vector + 1
a_vector

[1]  3  4  6  8  2 14

#remove the first element of our vector
a_vector <- a_vector[-1]
a_vector

[1]  4  6  8  2 14

We not only can extract the nth element but if we have header names then we can also use these to retrieve data:

#create a vector and give it names (i.e. for counts from some microbes)
x <- c(300, 410, 531)
names(x) <- c("Ecoli","Archaeoglobus","Ignicoccus")

#check how our data looks
x

        Ecoli Archaeoglobus    Ignicoccus 
          300           410           531 

#now we can retrieve part of the vector using the names
x[c("Ecoli","Ignicoccus")]

     Ecoli Ignicoccus 
       300        531 

Changing vectors

We can also change elements in our vector:

#create a vector
x <- 1:10

#change the second last positions to 5 and 9
x[9:10] <- c(5,9)

#check if this worked
x

 [1] 1 2 3 4 5 6 7 8 5 9

#we can not only add things, we can also remove this using the minus/**-** symbol
x[-3]

[1] 1 2 4 5 6 7 8 5 9

#if we want to remove more than one thing we can use the **c()**
x[-(4:10)]

[1] 1 2 3

Matrices are the R objects in which the elements are arranged in a two-dimensional rectangular layout. They contain elements of the same type. Although you can construct matrices with characters or logicals, matrices are generally used to store numeric data.

The basic syntax for creating a matrix is:

matrix(data, nrow, ncol, byrow, dimnames)

• data: input vector whose components become the data elements from the matrix.
• dnrow: number of rows to be created.
• dncol: number of columns to be created.
• dbyrow: logical. If FALSE,(the default) the matrix is filled by columns, otherwise the matrix is filled by rows.
• ddimnames: A ‘dimnames’ attribute for the matrix: NULL or a list of length 2 giving the row and column names respectively.

In contrast in a data frame (see below) the columns contain different types of data, while in a matrix all the elements are the same type of data. A matrix in R is like a mathematical matrix, containing all the same type of thing (usually numbers). R often but not always can use dataframes and a matrix used interchangably.

• Individual elements in a matrix can be printed using [row,column]. For example [2,3] would pull out the value in the 2nd ROW and third COLUMN.

• dim() is extremely useful to control whether our data was transformed correctly during different operations. For example, after we merge two files we would like to know that they still have the same number of rows as when we started the analysis. Same if we remove for example 10 samples, then we want to make sure that this is indeed what happened.

• head() is another useful function to check the first rows of a larger matrix (or dataframe)

• tail() same as head but showing the last rows

#define our row and column names
row.names = c("row1", "row2", "row3", "row4")
col.names = c("col1", "col2", "col3")

#create our matrix (check the help function to see what is happening)
matrix_A <- matrix(c(3:14), nrow = 4, byrow = T, dimnames = list(row.names,col.names))

#check how our matrix looks like
matrix_A

     col1 col2 col3
row1    3    4    5
row2    6    7    8
row3    9   10   11
row4   12   13   14

#print the value in the 2row and 3rd column
matrix_A[2,3]

[1] 8

#print the values in the 3rd column
matrix_A[,3]

row1 row2 row3 row4 
   5    8   11   14 

#print everything except the 1st row
matrix_A[-1,]

     col1 col2 col3
row2    6    7    8
row3    9   10   11
row4   12   13   14

#print everything except the 2nd column
matrix_A[,-2]

     col1 col3
row1    3    5
row2    6    8
row3    9   11
row4   12   14

#see the dimensions of matrix, i.e. the nr of rows and columns
dim(matrix_A)

[1] 4 3

#check the first rows of our matrix, since our data is small, everything is shown
head(matrix_A)

     col1 col2 col3
row1    3    4    5
row2    6    7    8
row3    9   10   11
row4   12   13   14

Sometimes you need to store data of different types. For example, if you are collecting cell counts, you might want to have cell counts (numeric), the microbes investigated (character), their status (logical, with TRUE for alive and FALSE for dead, …. This kind of data can be stored in lists. Lists are the R objects which contain elements of different types (numeric, strings, vectors, even another list, or a matrix).

A list is created using the list() function.

For example, the following variable x is a list containing copies of three vectors n, s, b.

#define our vectors
n = c(20, 30, 50) 
s = c("Ecoli", "Archaeoglobus", "Bacillus") 
b = c(TRUE, FALSE, TRUE) 

#combine the vectors in a list
our_list = list(counts=n, strain=s, status=b) 

#show our list
our_list

$counts
[1] 20 30 50

$strain
[1] "Ecoli"         "Archaeoglobus" "Bacillus"     

$status
[1]  TRUE FALSE  TRUE

#sublist the second element in a list
our_list[2]

$strain
[1] "Ecoli"         "Archaeoglobus" "Bacillus"     

#retrieve the 2nd and 3rd member of our list
our_list[c(2, 3)] 

$strain
[1] "Ecoli"         "Archaeoglobus" "Bacillus"     

$status
[1]  TRUE FALSE  TRUE

#we can also retrieve elements of a list if we know the name using two different ways:
our_list$strain

[1] "Ecoli"         "Archaeoglobus" "Bacillus"     

our_list[["strain"]]

[1] "Ecoli"         "Archaeoglobus" "Bacillus"     

In the last example we use the $ dollar symbol to extract data, i.e. to extract variables in a dataset (a matrix, list, dataframe). I.e. above the data we want to access is ‘our_list’ and the variable we want to extract is the strain.

Dataframes are tables in which each column contains values of one variable type and each row contains one set of values from each column. You can think of a data frame as a list of vectors of equal length. Most of our data very likely will be stored as dataframes.

A Dataframe usually follows these rules:

• The top line of the table, called the header, contains the column names.
• Column names (i.e. the header of our data) should be non-empty (if they are, R provides the object with default values).
• Row names should be unique
• Each column should contain the same number of data items
• Each horizontal line after the header is a data row, which begins with the name of the row, and then followed by the actual data.
• Each data member of a row is called a cell.

Importantly, most of the things we have learned before, i.e. how to subset data, apply here too.

The growth data that we have read into R will be used to explain how dataframes work.

Viewing data Dataframes

• We can use the brackets as before to extract certain rows or columns.
• We can use the dollar sign to again extract information as long as we know the column names. I.e. now we want to access the shoot fresh weight (FW_shoot_mg) in our ‘growth_data’ dataframe.
• colnames() allows us to access the column names, i.e. the headers
• rownames() allows us to access the rownames of our data (usually these are numbered if not specified otherwise while reading the table)
• dim() allows us to check the dimensions (i.e. the number of rows and columns). This is useful to regullary check, especially if we modified our data somehow.
• head() shows the first rows of our dataframe

#view our table
kable(head(timecourse), format='markdown')

#check how many rows and columns our data has
dim(timecourse)

[1] 627   5

#extract the data from the 2nd row
timecourse[2,]

   SampleID Nutrient Condition Timepoint Rootlength
2 P_MgCl_t0        P      MgCl        t0      1.849

#extract the first three columns
head(timecourse[,1:3])

   SampleID Nutrient Condition
1 P_MgCl_t0        P      MgCl
2 P_MgCl_t0        P      MgCl
3 P_MgCl_t0        P      MgCl
4 P_MgCl_t0        P      MgCl
5 P_MgCl_t0        P      MgCl
6 P_MgCl_t0        P      MgCl

#extract a column of our data using the column name
head(timecourse$Timepoint)

[1] t0 t0 t0 t0 t0 t0
Levels: t0 t1 t2 t3 t4 t5

#print our headers
colnames(timecourse)

[1] "SampleID"   "Nutrient"   "Condition"  "Timepoint"  "Rootlength"

#get the dimensions
dim(timecourse)

[1] 627   5

When we print the rownames, we see that we have numbers from 1-105. When reading in a table into R it is the default behaviour how rownames are generated. As a general rule, if you want o have other rownames, these must be unique.

Adding new columns to Dataframes

Below is a very basic way to add a new column (we name it newColumn) and fill all rows with the word comment

#expand a dataframe, functions data.frame or cbind (or see below)
timecourse$newColumn <- "comment"

#check if that worked
kable(head(timecourse), format='markdown')

There are more sophisticated ways to add columns based on conditions or even merge dataframes. Some of these we will discuss later.

If we read in our own data, we should check as what type of class our table is stored. We have several ways to do this:

• class() = determines as what kind of object is stored
• str() = display the internal structure of an R object.

#check what kind of data we have:
class(timecourse)

[1] "data.frame"

#check how are different parts of our data stored?
str(timecourse)

'data.frame':   627 obs. of  6 variables:
 $ SampleID  : Factor w/ 23 levels "noP_350_t0","noP_350_t1",..: 18 18 18 18 18 18 18 18 18 18 ...
 $ Nutrient  : Factor w/ 2 levels "noP","P": 2 2 2 2 2 2 2 2 2 2 ...
 $ Condition : Factor w/ 2 levels "350","MgCl": 2 2 2 2 2 2 2 2 2 2 ...
 $ Timepoint : Factor w/ 6 levels "t0","t1","t2",..: 1 1 1 1 1 1 1 1 1 1 ...
 $ Rootlength: num  1.54 1.85 1.21 1.83 1.59 ...
 $ newColumn : chr  "comment" "comment" "comment" "comment" ...

We see that

• our data is stored in a dataframe
• that the data stored in different formats, i.e. numeric and characters
• the first 3 columns are stored in something we call a factor.

Factors are data objects that are used to represent categorical data and store it in its different levels. They are an important class for statistical analysis and for plotting. Factors are stored as integers, and have labels associated with these unique integers. Once created, factors can only contain a pre-defined set values, known as levels. By default, R always sorts levels in alphabetical order.

• factor() allows us to create our own factor

#lets make a vector
Nutrients <- c("P", "P", "noP", "noP")

#lets make our own simple factor
Nutrients_factor <- factor(c("P", "P", "noP", "noP"))

#lets compare the vector and factor we generated
Nutrients

[1] "P"   "P"   "noP" "noP"

Nutrients_factor

[1] P   P   noP noP
Levels: noP P

When we check our factor, we see that R assigns one level to P and another level to noP. We can also see, that R sorts the levels in an alphabetical way, even though in the initial code we first had P before noP.

Checking the behaviour of factors

Now, lets check how factors behave.

• levels() = only prints the levels of a given factor. We can also run this on any column of our dataframe.
• nlevels() = check how many levels we have.
• While factors look (and often behave) like character vectors, they are actually integers under the hood, and you need to be careful when treating them like strings. We can test this by looking at what type of object we generated.

#only print the levels
levels(Nutrients_factor)

[1] "noP" "P"  

levels(growth_data$Condition)

[1] "MgCl"      "Strain101" "Strain230" "Strain28" 

#check how many levels we have
nlevels(Nutrients_factor)

[1] 2

#what class do we have
class(Nutrients_factor)

[1] "factor"

typeof(Nutrients_factor)

[1] "integer"

Ordering factor levels

For some thing, the order of things might matter and then we need to order the factors ourselves.

#check our levels
levels(Nutrients_factor)

[1] "noP" "P"  

#reorder levels
Nutrients_factor_reordered <- factor(Nutrients_factor, levels = c("P", "noP"))

#check our levels
levels(Nutrients_factor_reordered)

[1] "P"   "noP"

Converting factors

Sometimes you need to explicitly convert factors to either text or numbers. Or numbers to characters, etc. To do this, you use the functions as.character() or as.numeric().

#convert our factor to a character
Nutrients_characters <- as.character(Nutrients_factor)
Nutrients_characters

[1] "P"   "P"   "noP" "noP"

Removing NAs

NAs are generated when our data contains a missing value. This can become problematic for certain programs and we can decide to remove all NAs. The function to do this is is.na.

#create a vector that includes an NA
y <- c(1,2,3,NA,5)

#check whether we have NAs
is.na(y)

[1] FALSE FALSE FALSE  TRUE FALSE

#remove NAs from our data
y[!is.na(y)]

[1] 1 2 3 5

• Exclamation marks in R –> != is “not equal to.”
• The function is.na(z)gives a logical vector of the same size as z with value TRUE if and only if the corre-sponding element in z is NA.
• I.e. in this example we have FALSE FALSE FALSE TRUE FALSE
• When using y[!is.na(y)], retain the columns were is.na is False

#create a vector that includes an NA
x <- c(1,2,3,NA,5)

#check whether we have NAs
is.na(x)

[1] FALSE FALSE FALSE  TRUE FALSE

#remove NAs from our data
x[is.na(x)] <- 0

#check data
x

[1] 1 2 3 0 5

Replacing NAs in our dataframe

If we check our example dataframe, i.e. row 30, we will see that we have NAs. Le’s remove them from our dataframe for all further analyses.

#check for NAs
timecourse[30,]

    SampleID Nutrient Condition Timepoint Rootlength newColumn
30 P_MgCl_t0        P      MgCl        t0         NA   comment

#remove rows that contain NAs
timecourse_noNA <- timecourse[!is.na(timecourse$Rootlength), ]

#check the data
timecourse[30,]

    SampleID Nutrient Condition Timepoint Rootlength newColumn
30 P_MgCl_t0        P      MgCl        t0         NA   comment

timecourse_noNA[30,]

      SampleID Nutrient Condition Timepoint Rootlength newColumn
32 noP_MgCl_t0      noP      MgCl        t0      1.399   comment

This prints our session info (used R version, packages, etc) and is very important to store to reproduce our code for others

sessionInfo()

R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] kableExtra_1.1.0 knitr_1.29      

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.4        rstudioapi_0.11   xml2_1.2.5        magrittr_1.5     
 [5] hms_0.5.3         munsell_0.5.0     rvest_0.3.5       viridisLite_0.3.0
 [9] colorspace_1.4-1  R6_2.5.0          rlang_0.4.8       highr_0.8        
[13] httr_1.4.1        stringr_1.4.0     tools_3.5.0       webshot_0.5.2    
[17] xfun_0.15         htmltools_0.4.0   ellipsis_0.3.0    yaml_2.2.1       
[21] digest_0.6.27     tibble_3.0.4      lifecycle_0.2.0   crayon_1.3.4     
[25] bookdown_0.20     readr_1.3.1       vctrs_0.3.4       glue_1.4.2       
[29] evaluate_0.14     rmarkdown_2.3     stringi_1.5.3     compiler_3.5.0   
[33] pillar_1.4.4      scales_1.1.1      rmdformats_0.3.7  pkgconfig_2.0.3  

Notice: At the moment, we do not want to clean up the working directory, therefore, in our documentation we put a # in front of it. If you want to run the code just remove the #

#rm(list = ls()) 

One important features of R (and of any programming language) are logical operators. When R evaluates an expression containing logical operators, it will return either TRUE or FALSE. It follows a list of them:

#basic operators
1==1

[1] TRUE

2>1

[1] TRUE

1==1 & 1>2

[1] FALSE

(1==1& 1>2) | 5<6

[1] TRUE

#only print data for the normal P conditions
P_data <- timecourse_noNA[timecourse_noNA$Nutrient == "P", ]

#test that all went ok
dim(timecourse_noNA)

[1] 526   6

dim(P_data)

[1] 255   6

#we can also subset our data by numbers, i.e. only keep rows were roots are longer than 5cm
growth_data_10cm <- timecourse_noNA[timecourse_noNA$Rootlength > 5, ]
dim(growth_data_10cm)

[1] 153   6

#instead of keeping rows with roots longer than 5cm we can also remove them using *!*
growth_data_no10cm <- timecourse_noNA[!timecourse_noNA$Rootlength > 5, ]
dim(growth_data_no10cm)

[1] 373   6

Above you see a good example why dim() can very useful as it allows us to quickly check that what we do actually works. Always check afterwards that your function did what you expect you to do since as long as R is able to do something it will do it regardless if that might be different that you want R to do.

Apart from the elementary operations, all of the common arithmetic functions are available: log, exp, sin, cos, tan, sqrt, etc. Other useful functions one can use on vectors are:

We already have seen important ways to subset data:

1. Use of the index by using the square brackets

#subsetting rows and columns using the index
timecourse_noNA[1:3,2:4]

  Nutrient Condition Timepoint
1        P      MgCl        t0
2        P      MgCl        t0
3        P      MgCl        t0

2. Use of operators

#subset using operators (only print rows if the Nutrient column equals P)
P_data <- timecourse_noNA[timecourse_noNA$Nutrient == "P", ]
kable(head(P_data), format = "markdown", align = 'l', booktabs = T )

Now we want to look at some other examples:

grep and grepl

Oftentimes you may need to filter a data set based on a partial character string that is beyond the scope of comparison operators.

R provides such functions (grep and grepl) that match character patterns in specified vector. While both of these functions find patterns, they return different output types based on those patterns.

• grep returns numeric values that correspond to the indexed locations of the patterns
• grepl returns a logical vector in which “TRUE” represents a pattern match.

In our growth data, we only want to print measurements of our strains.

#grep return the index value of each matched pattern
grep("MgC", timecourse_noNA$Condition)

  [1]   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18
 [19]  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36
 [37]  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54
 [55]  55  56  57  58 119 120 121 122 123 124 125 126 127 128 129 130 131 132
 [73] 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150
 [91] 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168
[109] 169 170 171 172 173 174 175 176 177 178 231 232 233 234 235 236 237 238
[127] 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256
[145] 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274
[163] 275 276 277 278 279 280 281 282 283 284 285 286 287 346 347 348 349 350
[181] 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368
[199] 369 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433
[217] 434 435 436 437 438 439 440 441 505 506 507 508 509 510 511 512 513 514
[235] 515

#grepl returns a logical output for each indices in the original vector 
#with "TRUE" representing matched patterns
grepl("MgC", timecourse_noNA$Condition)

  [1]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
 [13]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
 [25]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
 [37]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
 [49]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE
 [61] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
 [73] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
 [85] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
 [97] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[109] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE
[121]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[133]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[145]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[157]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[169]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE
[181] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[193] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[205] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[217] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[229] FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[241]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[253]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[265]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[277]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[289] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[301] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[313] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[325] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[337] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE
[349]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[361]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE
[373] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[385] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[397] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[409] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE
[421]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[433]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE
[445] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[457] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[469] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[481] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[493] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[505]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE
[517] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE

Now lets use this to actually filter our data table for a pattern.

filter_for_value <-timecourse_noNA[grepl("MgC", timecourse_noNA$Condition),]

kable(filter_for_value) %>%
  kable_styling() %>%
  scroll_box(width = "700px", height = "400px")

#filter data set based on values that do not match the specified pattern (by using the minus symbol)
filter_for_not_a_value <- timecourse_noNA[-grepl("MgC", timecourse_noNA$Condition),]

kable(filter_for_not_a_value) %>%
  kable_styling() %>%
  scroll_box(width = "700px", height = "400px")